Working with the AI Assistant

Use Duet’s AI agent to assist with coding tasks

Each Duet room includes a dedicated AI assistant powered by Llama 3 8B. The AI can answer questions, run commands in a Cloudflare Sandbox, and help with collaborative coding tasks.

AI Features Overview

The AI assistant:

• Uses Meta’s Llama 3 8B Instruct model via Cloudflare AI (cf-worker/index.ts:123)
• Maintains conversation context (up to 50 messages per room)
• Can execute commands in a dedicated Cloudflare Sandbox
• Shares state across all users in the room via Durable Objects
• Automatically extracts and runs commands wrapped in <run> tags

AI Sidebar

The AI conversation appears in a sidebar on the right side of your screen.

Toggling the Sidebar

• ctrl+a - Show/hide the AI sidebar
• Sidebar only appears when window is at least 120 characters wide (internal/ui/model.go:21)

Scrolling the AI History

• ctrl+j - Scroll down
• ctrl+k - Scroll up
• Auto-scrolls to your most recent prompt after receiving a response

AI Sidebar Layout

The sidebar displays (internal/ui/model.go:116-140):

• Conversation history with role indicators (user/agent)
• Username for each user message
• Timestamp for messages
• Loading spinner when AI is processing
• Up to 10 recent messages used as context for inference

Sending AI Messages

1. Enter AI mode

   Press ctrl+g from the terminal.

   You’ll see a prompt: “Ask the AI…”

2. Type your question

   Enter your question or request. Examples:

   • “How do I parse JSON in Node.js?”
   • “Find all TODO comments in this project”
   • “Create a simple Express server”
3. Submit

   Press Enter to send the message.

   The AI will respond within a few seconds (30-second timeout).

4. Cancel (optional)

   Press Esc to cancel and return to normal terminal mode without sending.

See internal/ui/model.go:380-389 for the AI prompt implementation.

AI System Prompt

The AI is configured with this system prompt (cf-worker/index.ts:156-161):

“You are Duet, a concise pair-programming assistant. You can run commands in a sandbox using <run>command</run> tags. When asked to perform an action, briefly explain what you will do and wrap the exact shell command(s) in <run> tags. Do NOT include predicted output in your response - just provide the explanation and command.”

This means the AI will:

• Give concise, focused responses
• Automatically execute commands when appropriate
• Avoid verbose explanations

AI Command Execution

The AI can automatically run commands in a Cloudflare Sandbox when it includes <run> tags in its response.

How It Works

1. AI generates response with commands in <run> tags:

   I'll search for all JavaScript files in the project.
   <run>find . -name "*.js" -type f</run>
   

2. Commands are extracted and executed in the sandbox (cf-worker/index.ts:185-208)

3. Output is appended to the AI’s response:

   I'll search for all JavaScript files in the project.
   
   Output (find . -name "*.js" -type f):
   ./src/index.js
   ./src/utils.js
   ./test/app.test.js
   

4. Result is broadcast to all users in the room

Command Limitations

• Output is truncated to 500 characters (stdout or stderr)
• 30-second timeout per command
• Runs in an isolated Cloudflare Sandbox (not the shared terminal)
• One sandbox instance per room

Direct Sandbox Commands

You can also run commands directly in the Cloudflare Sandbox without AI assistance:

1. Enter sandbox mode

   Press ctrl+r from the terminal.

   You’ll see a prompt: “Command to run…”

2. Enter command

   Type the command you want to execute in the sandbox:

   ls -la /app
   
3. Submit

   Press Enter to execute.

   You’ll see a toast notification with the command and output summary.

See internal/ui/model.go:391-399 and 486-489 for implementation.

Sandbox vs Terminal

Feature	Shared Terminal	Cloudflare Sandbox
Access	All users see real-time	Only command output visible
Persistence	Workspace files persist	Isolated per-room sandbox
Tools	nvim, Node.js, git	Standard Unix utilities
Best for	Collaborative editing	AI-driven commands, quick checks

AI Message Format

Messages are structured as (cf-worker/index.ts:18-23):

interface DuetMessage {
  role: "user" | "agent";
  userId?: string;  // Username of sender
  text: string;     // Message content
  ts: number;       // Timestamp
}

AI Context & Memory

Conversation History

• Each room stores up to 50 messages (internal/ui/model.go:179)
• Last 10 messages are sent as context for each new request (cf-worker/index.ts:163)
• History is shared across all users via Durable Object state
• New users joining see full conversation history (internal/ui/model.go:228-230)

Syncing Across Users

• When one user sends an AI message, all users see the response
• Updates are broadcast via “ai_sync” events (internal/ui/model.go:205-208)
• AI sidebar automatically updates for all participants

Configuration Requirements

AI features require a Cloudflare Worker URL:

./duet --worker https://duet-cf-worker.example.workers.dev

If no worker URL is configured:

• ctrl+g shows: “AI not configured (no worker URL)”
• ctrl+r shows: “Sandbox not configured (no worker URL)”

See internal/ui/model.go:381-383 and 391-394.

Troubleshooting

”AI not configured”

The server wasn’t started with a --worker URL. AI features are disabled.

AI Response Timeout

AI requests timeout after 30 seconds (internal/ui/model.go:500). If this happens:

• Try a simpler/shorter prompt
• Check Cloudflare Worker logs for errors
• Verify the worker URL is accessible

Command Execution Fails

If sandbox commands fail:

• Check the error message in the AI response
• Verify the command syntax is correct
• Note that some system commands may not be available in the sandbox

AI Sidebar Too Small

• Increase your terminal window size
• Toggle the sidebar off with ctrl+a if you need more terminal space
• Minimum recommended window: 120x24 characters

Best Practices

Effective AI Prompts

• Be specific: “Parse this JSON file” → “Show me how to parse JSON with error handling in Node.js”
• One task at a time: AI works best with focused requests
• Use for exploration: Great for finding files, checking syntax, quick references

Collaboration

• All users see AI responses - communicate before asking questions that might clutter the conversation
• Use AI for repetitive tasks (finding files, boilerplate code)
• Remember: AI runs in sandbox, not the shared terminal

Resource Usage

• AI inference happens on Cloudflare’s edge network
• Sandbox execution is isolated per room
• Room cleanup automatically terminates sandbox and resets AI state (cf-worker/index.ts:244-266)